Floor plan image generating method and device, computer device and storage medium

ABSTRACT

A floor plan image generating methodology is provided. The methodology includes: acquiring a boundary of a target building and a layout constraint of the target building; outputting multiple first floor plan images according to the layout constraint of the target building; selecting multiple second floor plan images from the multiple first floor plan images; applying a layout constraint of each of the second floor plan images to the boundary of the target building, and obtaining a layout of the target building corresponding to each of the multiple second floor plan images; inputting the layout of the target building and the boundary of the target building into a floor plan image generating network; and obtaining a predicted floor plan image of the target building outputted by the floor plan image generating network.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of international patent applicationNo. PCT/CN2020/111335, filed on Aug. 26, 2020, which claims priority toChinese Patent Application No. 202010257143.6, filed on Apr. 3, 2020 andentitled “FLOOR PLAN IMAGE GENERATING METHOD AND DEVICE, COMPUTER DEVICEAND STORAGE MEDIUM”. The contents of the above identified applicationsare hereby incorporated herein in their entireties by reference.

TECHNICAL FIELD

The present application relates to the field of computer technology, andin particular, to a floor plan image generating method, a floor planimage generating device, a computer device, and a storage medium.

BACKGROUND

In daily life, a design for a building, such as a house type design forthe building or a layout design for the building, is often required.Since a manual design is often time-consuming, an automatic floor planimage generating method has emerged.

In the related automatic floor plan image generating method, a boundaryshape of the building is necessarily acquired first. Then a floor planimage, matched with the building boundary shape, is searched from alarge number of pre-stored floor plan image data, and the searched floorplan image is output.

However, the related automatic floor plan image generating method mayquickly generate a floor plan image, but quality of the generated floorplan image is not too high because the floor plan image is generatedonly by performing a simple search and match in a data set.

SUMMARY

In view of the above technical problems, it is necessary to provide afloor plan image generating method, a floor plan image generatingdevice, a computer device, and a storage medium that can improve thequality of the floor plan image.

In a first aspect, a floor plan image generating method is provided, andthe method includes: acquiring a boundary of a target building and alayout constraint of the target building, the layout constraintincluding room types, room quantities, room locations, and adjacencyrelations between rooms; outputting multiple first floor plan imagesaccording to the layout constraint of the target building; selectingmultiple second floor plan images from the multiple first floor planimages, and a matching degree between a boundary of each of the multiplesecond floor plan images and the boundary of the target buildingsatisfying a preset condition; for each of the second floor plan images,applying a layout constraint of each of the second floor plan images tothe boundary of the target building, and obtaining a layout of thetarget building corresponding to each of the multiple second floor planimages; and for each layout of the target building, inputting the layoutof the target building and the boundary of the target building into afloor plan image generating network, and obtaining a predicted floorplan image of the target building outputted by the floor plan imagegenerating network, wherein the predicted floor plan image includes aroom bounding box of each room in the target building and a positionalrelation of each room bounding box in the boundary of the targetbuilding.

In one of the embodiments, the outputting the multiple first floor planimages according to the layout constraint of the target buildingincludes retrieving the multiple first floor plan images that satisfythe layout constraint of the target building from a pre-stored floorplan image data set.

In one of the embodiments, the selecting the multiple second floor planimages from the multiple first floor plan images, includes: obtainingturning functions of boundaries of the multiple first floor plan imagesand a turning function of the boundary of the target building;calculating an accumulated difference between each of the turningfunctions of the boundaries of the multiple first floor plan images andthe turning function of the boundary of the target building; and usingthe multiple first floor plan images each corresponding to theaccumulated difference smaller than a preset difference threshold as themultiple second floor plan images.

In one of the embodiments, for each of the second floor plan images,applying the layout constraint of each of the second floor plan imagesto the boundary of the target building, and obtaining the layout of thetarget building corresponding to each of the second multiple floor planimages, includes: adjusting each of the multiple second floor planimages until an angle between a front door direction of each of themultiple second floor plan images and a front door direction of thetarget building is smaller than a preset angle threshold, and obtainingadjusted second floor plan images; and for each of the adjusted secondfloor plan images, applying room types, room quantities, room locations,and adjacency relations between rooms of each of the adjusted secondfloor plan images to the boundary of the target building, and obtainingthe layout of the target building corresponding to each of the multiplesecond floor plan images.

In one of the embodiments, the for each layout of the target building,inputting the layout of the target building and the boundary of thetarget building into the floor plan image generating network, andobtaining the predicted floor plan image of the target buildingoutputted by the floor plan image generating network, includes: for eachlayout of the target building, inputting the layout of the targetbuilding into an image neural network, and obtaining a room featurevector corresponding to each room in the layout of the target building;inputting the boundary of the target building into a first convolutionalneural network (CNN), and obtaining a boundary feature vector of thetarget building; correlating the room feature vector corresponding toeach room with the boundary feature vector to obtain a correlatedfeature vector corresponding to each room; inputting the correlatedfeature vector corresponding to each room into a first multilayerperceptron (MLP), and obtaining an initial bounding box corresponding toeach room in the layout of the target building; mapping the correlatedfeature vector corresponding to each room by using the initial boundingbox corresponding to each room to obtain a first feature imagecorresponding to each room; combining multiple first feature images ofmultiple rooms to obtain a second feature image, and inputting thesecond feature image into a second CNN to obtain a grid floor plan imageof the target building; and inputting multiple correlated featurevectors of the multiple rooms, multiple initial bounding boxes of themultiple rooms and the grid floor plan image into a bounding boxoptimizing network to obtain the predicted floor plan image of thetarget building, and the bounding box optimizing network comprising athird CNN, a pooled layer of a region of interest, and a second MLP.

In one of the embodiments, the method further includes: obtaining atraining data set, and the training data set including multiple floorplan image data; training an initial floor plan image generating networkby using the training data set to obtain trained floor plan imagegenerating network; calculating a loss value of the trained floor planimage generating network by using a cross entropy loss function, aregression loss function, and a geometric loss function; and adjustingparameters of the trained floor plan image generating network accordingto the loss value to obtain the floor plan image generating network.

In one of the embodiments, the method further includes: aligning eachroom bounding box of the target building with the boundary of the targetbuilding; and aligning adjacent room bounding boxes of the targetbuilding.

In a second aspect, a floor plan image generating device is provided andincludes a processor and a memory. The memory has computer programsstored therein, and the computer programs, when being executed by theprocessor, causes the processor to perform: acquiring a boundary of atarget building and a layout constraint of the target building, thelayout constraint comprising room types, room quantities, roomlocations, and adjacency relations between rooms; outputting multiplefirst floor plan images according to the layout constraint of the targetbuilding; selecting multiple second floor plan images from the multiplefirst floor plan images, and a matching degree between a boundary ofeach of the multiple second floor plan images and the boundary of thetarget building satisfying a preset condition; for each of the secondfloor plan images, applying a layout constraint of each of the secondfloor plan images to the boundary of the target building, and obtaininga layout of the target building corresponding to each of the multiplesecond floor plan images; and for each layout of the target building,inputting the layout of the target building and the boundary of thetarget building into a floor plan image generating network, andobtaining a predicted floor plan image of the target building outputtedby the floor plan image generating network. The predicted floor planimage includes a room bounding box of each room in the target buildingand a positional relation of each room bounding box in the boundary ofthe target building.

In a third aspect, a computer device is provided, a memory and aprocessor. The memory stores computer programs, and the processor, whenexecuting the computer programs, performs: acquiring a boundary of atarget building and a layout constraint of the target building, thelayout constraint comprising room types, room quantities, roomlocations, and adjacency relations between rooms; outputting multiplefirst floor plan images according to the layout constraint of the targetbuilding; selecting multiple second floor plan images from the multiplefirst floor plan images, and a matching degree between a boundary ofeach of the multiple second floor plan images and the boundary of thetarget building satisfying a preset condition; for each of the secondfloor plan images, applying a layout constraint of each of the secondfloor plan images to the boundary of the target building, and obtaininga layout of the target building corresponding to each of the multiplesecond floor plan images; and for each layout of the target building,inputting the layout of the target building and the boundary of thetarget building into a floor plan image generating network, andobtaining a predicted floor plan image of the target building outputtedby the floor plan image generating network. The predicted floor planimage includes a room bounding box of each room in the target buildingand a positional relation of each room bounding box in the boundary ofthe target building.

In a fourth aspect, a non-transitory computer-readable storage medium isprovided. Computer programs are stored on the non-transitorycomputer-readable storage medium, and the computer programs, when beingexecuted by a processor, perform: acquiring a boundary of a targetbuilding and a layout constraint of the target building, the layoutconstraint including room types, room quantities, room locations, andadjacency relations between rooms; outputting multiple first floor planimages according to the layout constraint of the target building;selecting multiple second floor plan images from the multiple firstfloor plan images, and a matching degree between a boundary of each ofthe multiple second floor plan images and the boundary of the targetbuilding satisfying a preset condition; for each of the second floorplan images, applying a layout constraint of each of the second floorplan images to the boundary of the target building, and obtaining alayout of the target building corresponding to each of the multiplesecond floor plan images; and for each layout of the target building,inputting the layout of the target building and the boundary of thetarget building into a floor plan image generating network, andobtaining a predicted floor plan image of the target building outputtedby the floor plan image generating network. The predicted floor planimage includes a room bounding box of each room in the target buildingand a positional relation of each room bounding box in the boundary ofthe target building.

The floor plan image generating method and device, the computer device,and the storage medium above are implemented by means of: acquiring aboundary of a target building and a layout constraint of the targetbuilding, the layout constraint including room types, room quantities,room locations, and adjacency relations between rooms; then, outputtingmultiple first floor plan images according to the layout constraint ofthe target building; then, selecting multiple second floor plan imagesfrom the multiple first floor plan images, and a matching degree betweena boundary of each of the multiple second floor plan images and theboundary of the target building satisfying a preset condition; for eachof the second floor plan images, applying a layout constraint of each ofthe second floor plan images to the boundary of the target building, andobtaining a layout of the target building corresponding to each of themultiple second floor plan images; and finally, for each layout of thetarget building, inputting the layout of the target building and theboundary of the target building into a floor plan image generatingnetwork, and obtaining a predicted floor plan image of the targetbuilding outputted by the floor plan image generating network, whereinthe predicted floor plan image includes a room bounding box of each roomin the target building and a positional relation of each room boundingbox in the boundary of the target building. In the floor plan imagegenerating method provided by the present application, during thegeneration of the floor plan image, not only the boundary of thebuilding, but also the layout constraint of the building is fullyconsidered, therefore the predicted floor plan image finally generatedmeets the actual needs better, thereby improving the quality of thegenerated floor plan image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic flowchart of a floor plan image generating methodof an embodiment.

FIG. 2 is a schematic flowchart of a method for selecting multiplesecond floor plan images from multiple first floor plan images accordingto an embodiment of the floor plan image generating method.

FIG. 3 is a schematic flowchart of a method for obtaining a layout of atarget building corresponding to each second floor plan image for eachsecond floor plan image, by applying a layout constraint of the secondfloor plan image to a boundary of the target building, according to anembodiment of the floor plan image generating method.

FIG. 4 is a schematic flowchart of a method for obtaining a predictedfloor plan image of the target building outputted by a floor plan imagegenerating network for each layout of the target building, by inputtingthe layout of the target building and the boundary of the targetbuilding into the floor plan image generating network, according toanother embodiment of the floor plan image generating method.

FIG. 5 is a schematic view showing a specific structure of a floor planimage generating network according to an embodiment.

FIG. 6 is a schematic view showing a specific structure of a boundingbox optimizing network according to an embodiment.

FIG. 7 is a schematic flowchart of a method for training the floor planimage generating network according to an embodiment of the floor planimage generating method.

FIG. 8 is a schematic overall flowchart of an embodiment of the floorplan image generating method.

FIG. 9 is schematic view showing the boundary of the target building anda turning function according to an embodiment.

FIG. 10 shows schematic views illustrating alignment and adjustment of apredicted floor plan image according to an embodiment of the floor planimage generating method.

FIG. 11 shows a group of views illustrating exemplary results generatedby different boundary inputs and different user constraint conditionsaccording to an embodiment of the floor plan image generating method.

FIG. 12 is a structural block view showing a floor plan image generatingdevice of an embodiment.

FIG. 13 is a structural block view showing a floor plan image generatingdevice of another embodiment.

FIG. 14 is a view showing an internal structure of a computer device ofan embodiment.

DETAILED DESCRIPTION OF THE EMBODIMENTS

In order to make the objective, technical solutions and advantages ofthe present application clearer and better understood, the presentapplication will be further described in detail with reference to theaccompanying drawings and embodiments. It should be understood that theembodiments described hereinafter are only used to explain the presentapplication, but not intended to limit the present application.

In an embodiment of the present application, as shown in FIG. 1 , afloor plan image generating method is provided. Taking the methodapplied to a terminal as an example, the terminal may be, but is notlimited to, any one of various personal computers, notebook computers,smartphones, panel computers, and portable wearable devices, and themethod includes the following step 101 to step 105.

At step 101, the terminal acquires a boundary of a target building and alayout constraint of the target building, and the layout constraintincludes room types, room quantities, room locations, and adjacencyrelations between rooms.

The boundary of the target building mentioned in this step refers to anoutline shape of the house type of the target building. Optionally, auser may manually input the boundary of the target building into theterminal, or a measuring device may measure the target building and sendacquired outline data of the house type to the terminal. In anembodiment, the terminal has a measurement function, and may directlymeasure the target building to acquire the boundary of the targetbuilding.

The layout constraint of the target building is inputted into theterminal by the user according to actual needs. Optionally, a dialog boxmay be displayed on the terminal, so that the user may input the desiredroom types, room quantities, room locations, adjacency relations betweenrooms, and other information into the dialog box of the terminal. Inaddition, the terminal may display the boundary of the target buildingto the user, and the user may roughly draw the desired room types, roomquantities, room locations, and the adjacency relations between rooms inthe boundary of the target building by means of an external device suchas a mouse.

At step 102, the terminal outputs multiple first floor plan imagesaccording to the layout constraint of the target building.

At this step, after acquiring the layout constraint of the targetbuilding, the terminal outputs the first floor plan images that meetsthe layout constraint required by the user. In an embodiment, the samelayout constraint may have multiple layout forms. To achieve an aimedaccuracy, the room quantities, the room types, the room locations andthe adjacency relations between rooms of these first floor plan imagesmay fully meet the user's requirements. However, in some possible cases,in order to provide various floor plan images as possible, the layoutconstraint of the outputted first floor plan images may also partiallysatisfy the user's requirements. For example, the number of the roomsrequired by the user is four, and the number of the rooms in theoutputted first floor plan image may be three or five. The matchingdegree between the layout constraint of the first floor plan image andthe layout constraint inputted by the user may be preset.

At step 103, the terminal selects multiple second floor plan images fromthe multiple first floor plan images, and a matching degree between aboundary of each of the multiple second floor plan images and theboundary of the target building satisfies a preset condition.

The layout constraint of the first floor plan image meets therequirement of the user, but the boundary of the first floor plan imagemay not necessarily meet the boundary of the target building. Therefore,a selection needs to be performed again to obtain the floor plan images,whose layout constraint and boundary both meet the requirements, fromthe first floor plan images.

In an embodiment, a method for judging the matching degree between theboundary of the first floor plan image and the boundary of the targetbuilding at this step may include: overlapping and comparing an areaenclosed by the boundary of the first floor plan image and an areaenclosed by the boundary of the target building, calculating a ratio ofan area of an overlap region to the area enclosed by the boundary of thetarget building, and selecting the first floor plan image to be thesecond floor plan image if the ratio exceeds a preset area threshold. Inan embodiment, at the terminal, each of the first floor plan images isdisplayed in the boundary of the target building inputted by the user,so that the user can intuitively analyze the matching degree between thefirst floor plan image and the boundary and select the second floor planimage.

In addition, the method for judging the matching degree between theboundary of the first floor plan image and the boundary of the targetbuilding may also include: calculating a perimeter of a boundary and thenumber of corners of the first floor plan image and a perimeter of theboundary and the number of corners of the target building, thenselecting the first floor plan image to be the second floor plan image,if a difference between the perimeter of the boundary of the first floorplan image and the perimeter of the boundary of the target building isless than a preset perimeter difference threshold, and if a differencebetween the number of corners of the first floor plan image and thenumber of corners of the target building is less than a preset numberdifference threshold.

In an embodiment, a turning function of the boundary of the first floorplan image and a turning function of the boundary of the target buildingmay be calculated, then the two turning functions are compared, and thesecond floor plan image may be selected from the first floor plan imageaccording to a result of comparison.

At Step 104, for each of the second floor plan images, the terminalapplies a layout constraint of each of the second floor plan images tothe boundary of the target building, and obtains a layout of the targetbuilding corresponding to each of the multiple second floor plan images.

After the second floor plan image whose layout constraint and boundaryboth meet the user's requirements is selected, the layout of the secondfloor plan image may be directly applied to the boundary of the targetbuilding. Under the same layout constraint, there may be various formsof floor plan images, so the selected second floor plan images may havemultiple forms of layouts. Between the floor plan images with the sameor similar boundaries, the applicability is stronger after the layoutconstraint is directly used, therefore the layout constraint of thesecond floor plan image may be directly applied to the boundary of thetarget building to obtain various layouts of the target building.

At Step 105, for each layout of the target building, the terminal inputsthe layout of the target building and the boundary of the targetbuilding into a floor plan image generating network, and obtains apredicted floor plan image of the target building outputted by the floorplan image generating network. The predicted floor plan image includes aroom bounding box of each room in the target building and a positionalrelation of each room bounding box in the boundary of the targetbuilding.

The multiple layouts of the target buildings obtained at the step aboveare multiple layouts just applied to the boundary of the targetbuilding, and specific floor plan images of the target building is notformed. Therefore, each layout of the target building is necessarilyinputted into the floor plan image generating network to obtain moreaccurate floor plan images. The multiple predicted floor plan imagesobtained at this step may be provided for the user to select, andprovided for guiding the user to lay out the house type of the targetbuilding. At this step, the predicted floor plan images may be presentedby the terminal in a visual display, and the displayed contents includedisplaying a boundary shape of the target building and displaying roombounding box of each room at the corresponding position in the boundaryshape.

In the floor plan image generating method above, the boundary of thetarget building and the layout constraint of the target building areacquired, and the layout constraint includes the room types, the roomquantities, the room locations, and the adjacency relations betweenrooms. Then the multiple first floor plan images are provided accordingto the layout constraint of the target building. Then, the multiplesecond floor plan images are selected from the multiple first floor planimages, and the matching degree between the boundary of each of themultiple second floor plan images and the boundary of the targetbuilding satisfies the preset condition. For each of the second floorplan images, the layout constraint of each of the second floor planimages is applied to the boundary of the target building, and the layoutof the target building corresponding to each second floor plan image isobtained. Finally, for each of the layouts of the target building, theeach of the layouts of the target building and the boundary of thetarget building are inputted into the floor plan image generatingnetwork, and the predicted floor plan image of the target buildingoutputted by the floor plan image generating network are obtained. Thepredicted floor plan image includes the room bounding box of each roomin the target building and the positional relation of each room boundingbox in the boundary of the target building. In the floor plan imagegenerating method provided by the present application, during thegeneration of the floor plan image, not only the boundary of thebuilding, but also the layout constraint of the building is fullyconsidered, therefore the predicted floor plan image finally generatedmeets the actual needs better, thereby improving the quality of thegenerated floor plan image.

In this embodiment of the present application, the outputting themultiple first floor plan images according to the layout constraint ofthe target building includes the following step.

The multiple first floor plan images that satisfy the layout constraintof the target building are retrieved from a pre-stored floor plan imagedata set.

Optionally, the terminal may pre-store a large number of floor planimage data, then encode and store the large number of floor plan imagedata. The advantage of encoding is that, in subsequent steps, the firstfloor plan image corresponding to the layout constraint of the targetbuilding inputted by the user may be retrieved quickly

In an embodiment, the way of encoding the floor plan image may beencoding the room types, the room quantities, the room locations and theadjacency relations between rooms in the floor plan image.

The way of coding will be described in detail herein. First, each roomin the floor plan image may be represented by a node, and if two roomshave an adjacency relationship in the floor plan image, an edge is addedbetween the nodes corresponding to the rooms. The room types are dividedinto 13 types, including living room, master bedroom, secondary bedroom,guest room, child room, reading room, dining room, bathroom, kitchen,balcony, storage room, wardrobe and antechamber. The room types may beencoded in sequence with numerals 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12,and 13. Then, in order to encode the room locations, the boundary of thetarget building may be replaced with a rectangular bounding box to getthe bounding box of the target building, and then the rectangularbounding box is divided into a K×K grid, where K is a positive integergreater than or equal to 1. A grid coordinate where a center of eachroom in the floor plan image is located is used as the code of the roomposition. For the adjacency relation between rooms, all interior doorsin the floor plan image are found first, then rooms on both sides of adoor is taken as an adjacent room pair. Next, it may be examined whethera distance between any two rooms is less than a given threshold relativeto the room bounding box, that is, it may be determined whether there isan adjacency relation of sharing a wall between two rooms. For each pairof adjacent rooms, the edge connecting the two rooms is encoded, and thecode may be represented by left, right, upper, lower, upper left, upperright, lower left, or lower right, which represents a relativepositional relationship between two rooms. Since the relative positionalrelationship between rooms is relative, for example, taking B as areference, if A is over B, the edge connecting A and B may be encoded as“upper”, but if A is taken as a reference, and B is below A, then theedge connecting A and B may be encoded as “lower”. Although the twocodes are different formally, they are the same essentially. Therefore,in an actual process of encoding, two relative codes may be stored foran edge at the same time, or one relative code may be randomly selectedto be stored. In addition, the room sizes may also be encoded, and aratio of an area of a room to the entire area of the floor plan image isused as a code of a room size.

The coding method above may be applied to the layout constraint inputtedby the user. After the layout constraint inputted by the user is encodedby using the same means of coding, the first floor plan images that meetthe user's requirements may be selected by means of relatively simplecodes, thereby greatly improving the efficiency of selecting the firstfloor plan images by the terminal.

In an embodiment of the present application, referring to FIG. 2 , inthe floor plan image generating method, a method for selecting multiplesecond floor plan images from multiple first floor plan images includesthe following step 201 to step 203.

At Step 201, the terminal obtains turning functions of boundaries of themultiple first floor plan images and a turning function of the boundaryof the target building.

The turning function may convert a two-dimensional polygon of theboundary into a one-dimensional contour line segment to achieve thepurpose of reducing dimensions, so as to facilitate a comparison of thematching degree of boundaries. The one-dimensional contour line segmentis located in a coordinate system formed by a normalized accumulatedboundary length and an accumulated angle. The turning function mayrecord a cumulated angle value of each turning point (corner) in thetwo-dimensional polygon of the boundary.

In an embodiment, starting from a front door of the floor plan image,the accumulated angle value of each turning point (corner) in thetwo-dimensional polygon of the boundary may be recorded clockwise. In apossible case, even for two buildings with the exact same boundary, adifferent front door position will lead to significant changes in thefloor plan image, which means that the front door positions need to beconsidered when two boundaries are compared, so the turning function ofthe target building and the turning function of the first floor planimage are recorded starting from the position of the front door.

At Step 202, the terminal calculates an accumulated difference betweeneach of the turning functions of the boundaries of the multiple firstfloor plan images and the turning function of the boundary of the targetbuilding.

After the turning function of the boundary of the target building andthe turning function of the first floor plan image are obtained, theaccumulated difference between the two turning functions needs to becalculated. In an embodiment, an area enclosed by the difference betweenthe two turning functions and a normalized accumulated boundary lengthaxis of the coordinate system may be used as the accumulated difference.

At Step 203, the terminal uses the multiple first floor plan images eachcorresponding to the accumulated difference smaller than a presetdifference threshold as the multiple second floor plan images.

The smaller the accumulated difference between the turning function ofthe boundary of the target building and the turning function of theboundary of the first floor plan image, the higher a similarity betweenthe boundary of the target building and the boundary of the first floorplan image. At this step, the preset difference threshold is set toselect the second floor plan images. In this case, the boundary of theselected second floor plan image is still a little different from theboundary of the target building, but the difference is within anacceptable range.

In addition, if it is desired to obtain the second floor plan image thatexactly matches the boundary of the target building, the first floorplan image with a zero accumulated difference may be selected.

In the embodiment of the present application, by using the turningfunction, a comparison of the two-dimensional boundaries is convertedinto a mathematical calculation of the one-dimensional contour linesegments, thereby greatly improving the efficiency and the accuracy ofselecting the second floor plan image.

In an embodiment of the present application, referring to FIG. 3 , inthe floor plan image generating method, a method for obtaining thelayout of the target building corresponding to each second floor planimage by applying the layout constraint of the second floor plan imageto the boundary of the target building for each second floor plan imageincludes the following step 301 to step 302.

At Step 301, the terminal adjusts each of the second floor plan imagesuntil an angle between a front door direction of each of the secondfloor plan images and a front door direction of the target building issmaller than a preset angle threshold, and adjusted second floor planimages are obtained.

At this step, before applying the layout constraint of the second floorplan image to the boundary of the target building, the second floor planimage needs to be adjusted, so that the boundary of the adjusted secondfloor plan image and the boundary of the target building overlap withthe greatest ratio. Because the boundary overlap ratio is calculatedbased on the turning function starting from the front door, so the frontdoor is used as a reference point for alignment. First, the front doorsof the two boundaries are aligned, which also prevents the front doorsfrom being blocked by any room. The criterion for judging whether thefront doors are aligned is to judge whether an angle between the frontdoor direction of a rotated second floor plan image and the front doordirection of the target building is smaller than the preset angle, suchas 45 degrees, etc. The front door direction mentioned in thisapplication is a vector connecting a center of the building bounding boxto a center of the door, or a vector connecting the center of thebounding box of the rotated second floor plan image to the front door.

It should be noted that, in the process of rotating the second floorplan image, the bottom edge of the rotated second floor plan image mustbe kept horizontal, and the perpendicular edges should be kept verticalas much as possible. Only such a floor plan image is a valid floor planimage.

At Step 302, for each of the adjusted second floor plan images, theterminal applies room types, room quantities, room locations, andadjacency relations between rooms of each of the adjusted second floorplan images to the boundary of the target building, and obtains thelayout of the target building corresponding to each of the multiplesecond floor plan images.

After the second floor plan image is adjusted, the layout constraint ofthe adjusted second floor plan image is transferred into the boundary ofthe target building. The way of transfer may be a full replication, thatis, the rooms of the target building are completely laid out accordingto the room types, room quantities, room locations, and adjacencyrelations between rooms in the adjusted second floor plan image.

However, since the boundary of the adjusted second floor plan image isstill different from the boundary of the target building, after thelayout constraint of the adjusted second floor plan image is directlyapplied to the boundary of the target building, some room nodes may belaid outside the boundary, which obviously does not meet the actualrequirements. Therefore, the obtained each layout of the target buildingmay be readjusted to ensure that all room nodes fall in the boundary ofthe target building. In an embodiment, the way of adjusting the nodeoutside the boundary may include following steps: establishing abounding box of the target building and grids in the bounding boxfirstly, then searching grid where the node outside the boundary islocated, further moving the node outside the boundary into the nearestgrid within the boundary. If there has been a node already in thenearest grid within the boundary, the existing node in the grid is movedto a new grid in the same direction. If a node is moved in the samedirection to the last grid within the boundary, and there still has beenanother node in the last grid, then two nodes are kept in the last grid.The same direction above refers to the same direction along which thenode outside the boundary is moved into the nearest grid within theboundary.

The way of adjusting the node outside the boundary above may be executedby the terminal automatically or by editing the multiple layouts of thetarget building by the user himself/herself after the terminal displaysthe multiple layouts of the target building.

In the embodiment of the present application, the second floor planimages are adjusted, the layout constraint of each of the adjustedsecond floor plan images is applied to the boundary of the targetbuilding, and the node outside the boundary is further adjusted, so thatthe layout of the target building obtained finally is more accurate andmeets the actual needs better.

In an embodiment of the present application, referring to FIG. 4 , inthe floor plan image generating method, a method for obtaining thepredicted floor plan image of the target building outputted by the floorplan image generating network for each layout of the target building, byinputting the layout of the target building and the boundary of thetarget building into the floor plan image generating network includesthe following step 401 to step 407.

At Step 401, for each layout of the target building, the terminal inputsthe layout of the target building into an image neural network (NN), andobtains a room feature vector corresponding to each room in the layoutof the target building.

At Step 402, the terminal inputs the boundary of the target buildinginto a first convolutional neural network (CNN) and obtains a boundaryfeature vector of the target building.

At Step 403, the terminal correlates the room feature vectorcorresponding to each room with the boundary feature vector to obtain acorrelated feature vector corresponding to each room.

At Step 404, the terminal inputs the correlated feature vectorcorresponding to each room into a first multilayer perceptron (MLP), andobtains an initial bounding box corresponding to each room in the layoutof the target building.

At Step 405, the terminal maps the correlated feature vectorcorresponding to each room by using the initial bounding boxcorresponding to each room, to obtain a first feature imagecorresponding to each room.

At Step 406, the terminal combines multiple first feature images ofmultiple rooms to obtain a second feature image, then inputs the secondfeature image into a second CNN to obtain a grid floor plan image of thetarget building.

At Step 407, the terminal inputs the multiple correlated feature vectorsof the multiple rooms, the multiple initial bounding boxes of themultiple rooms and the grid floor plan image into the bounding boxoptimizing network to obtain the predicted floor plan image of thetarget building, and the bounding box optimizing network includes athird CNN, a pooled layer of a region of interest (ROI), and a secondMLP.

The above step 401 to step 407 will be described with reference to aspecific structure of the floor plan image generating network of FIG. 5. The layout constraint and the boundary of the target building in FIG.5 are inputted into different neural networks respectively to obtain thecorrelated feature vector corresponding to each room. The MLP may beused to predict the initial bounding box (represented by B_(i) ⁰ in FIG.5 , where i denotes any positive integer between 0 and n) correspondingto each room within the boundary, map the correlated feature vectorcorresponding to each room by using the initial bounding boxcorresponding to each room to obtain the first feature imagecorresponding to each room, combine the obtained multiple first featureimages of the multiple rooms to obtain the second feature image, theninput the second feature image into the second CNN to obtain the gridfloor plan image. After the grid floor plan image is obtained,previously obtained initial bounding boxes may be optimized by abounding box optimizing network to obtain more accurate and optimizedbounding boxes (represented by B_(i) ⁰ in FIG. 5 , where i denotes anypositive integer between 0 and n). In the whole process, the outputs ofthe floor plan image generating network, including the initial boundingbox corresponding to each room, the grid floor plan image, and theoptimized bounding box corresponding to each room, may be obtained. Theoptimized bounding boxes may be combined to serve as the predicted floorplan image of the target building.

A specific structure of the bounding box optimizing network above mayrefer to FIG. 6 . As shown in FIG. 6 , the bounding box optimizingnetwork includes the third CNN, the pooled layer of the ROI, and thesecond MLP. First, the grid floor plan image is processed by the thirdCNN to obtain a third feature image, then the initial bounding boxcorresponding to each room and the third feature image are inputted intothe pooled layer of the ROI to obtain a feature vector with a specificlength corresponding to each room. Then the feature vector with thespecific length corresponding to each room and the correlated featurevector corresponding to each room are merged and inputted into thesecond MLP to obtain an optimized bounding box corresponding to eachroom. Further, a visual position relationship of the room bounding boxof each room within the boundary of the target building may be predictedaccording to the optimized bounding box of each room.

In the embodiment of the present application, the floor plan imagegenerating network may be formed by using a combination of multipletypes of networks to generate the predicted floor plan image, whichmakes the predicted floor plan image more accurate and have higherquality.

In an embodiment of the present application, referring to FIG. 7 , inthe floor plan image generating method, a method for training the floorplan image generating network includes the following step 701 to step704.

At Step 701, the terminal obtains a training data set, and the trainingdata set includes multiple floor plan image data.

At this step, a large number of floor plan images may be obtained from alarge number of resources on the Internet. After a large number of floorplan images are obtained, the obtained large number of floor plan imagesmay be preprocessed. The preprocessing includes converting color floorplan images into grayscale images, adjusting resolutions of all floorplan images to the same resolution, adjusting all floor plan images tohave the same size, or encoding all floor plan images by using theencoding method described above, etc. After the collected floor planimages are preprocessed, the processed floor plan images may be stored.In the process of training the floor plan image generating network, allnetworks included in FIG. 5 are actually trained.

At Step 702, the terminal trains an initial floor plan image generatingnetwork by using the training data set to obtain trained floor planimage generating network.

At Step 703, the terminal calculates a loss value of the trained floorplan image generating network by using a cross entropy loss function, aregression loss function, and a geometric loss function.

At Step 704, the terminal adjusts parameters of the trained floor planimage generating network according to the loss value to obtain the floorplan image generating network.

In order to ensure that the trained floor plan image generating networkhas good performance, multiple types of loss functions are used in thisstep to calculate the loss value of the floor plan image generatingnetwork.

The loss value is calculated by using the following equation:

L=L _(pix)(I)+L _(reg)({B _(i) ⁰})+L _(geo)({B _(i) ⁰})+L _(reg)({B _(i)¹})

Where, L_(pix)(I) denotes an image loss of the cross entropy of thepredicted floor plan image and the actual floor plan image,L_(reg)({B_(i) ⁰}) and L_(reg)({B_(i) ¹}) denote regression losses,L_(geo)({B_(i)}) denotes a geometric loss. The geometric loss is onlyapplicable to the initial bounding box, and the geometric loss of theinitial bounding box B_(i) ⁰ is defined as:

L _(geo)({B _(i) ⁰})=L _(coverage)({B _(i) ⁰})+L _(interior)({B _(i)⁰})+L _(mutex)({B _(i) ⁰})+L _(match)({B _(i) ⁰})

Where, L_(coverage) and L_(interior) both constrain a spatialconsistency between the boundary and the room bounding box, L_(mutex)constrains a spatial consistency between any two room bounding boxes,and L_(match)({B_(i) ⁰}) ensure that a predicted bounding box matches anactual bounding box. The first three terms ensure that the room boundingboxes properly cover the interior of the building. The last term, whichmakes a comparison between the predicted bounding box with the actualbounding box, and ensures that, during a training, the prediction of aposition and a size of the room bounding box is also improved.

Before providing more detailed information about the terms of thegeometric loss, firstly, two distance functions d_(in)(p, B) andd_(out)(p, B) are defined, and d_(in)(p, B) denotes a distance between apoint p and the inside of the bounding box B, and d_(out)(p, B) denotesa distance between the point p and the outside of the bounding box B.The point p is an arbitrary point, and d_(in)(p, B) and d_(out)(p, B)are as follows:

${d_{in}( {p,B} )} = \{ {{\begin{matrix}{0,} & {\ {{{{if}p} \in {\Omega_{in}(B)}};}} \\{{\min\limits_{q \in {\Omega_{bd}(B)}}{{p - q}}},\ } & {otherwise}\end{matrix}{d_{out}( {p,B} )}} = \{ \begin{matrix}{0,} & {\ {{{{if}p} \notin {\Omega_{in}(B)}};}} \\{{\min_{q \in {\Omega_{bd}(B)}}{{p - q}}},\ } & {otherwise}\end{matrix} } $

Where, a set Ω_(in)(B) and a set Ω_(bd)(B) represent pixel points insidethe bounding box B and pixel points of an outer boundary, respectively.

For a coverage loss, the boundary of the target building should becompletely covered by the combination of all room bounding boxes. In anembodiment, any point P∈Ω_(in)(B) should be covered by at least one roombounding box, therefore, the coverage loss is defined as follows:

${L_{coverage}( \{ B_{i} \} )} = \frac{\sum_{p \in {\Omega_{in}(B)}}{\min_{i}{d_{in}( {p,B} )}^{2}}}{❘{\Omega_{in}(B)}❘}$

Where, |Ω_(in)(B)| denotes the number of pixels in Ω_(in)(B).

For an internal loss, each room bounding box should be located withinthe target building bounding box ({circumflex over (B)}). Therefore, theinternal loss may be defined as:

${L_{interior}( \{ B_{i} \} )} = \frac{\sum_{i}{\sum_{p \in {\Omega_{in}(B_{i})}}{d_{in}( {p,\hat{B}} )}^{2}}}{\sum_{i}{❘{\Omega_{in}( B_{i} )}❘}}$

For a mutex loss, the overlap between room bounding boxes should be assmall as possible, such that the rooms may be compactly distributedinside the target building. Therefore, the mutex loss may be defined as:

${L_{mutex}( \{ B_{i} \} )} = \frac{\sum_{i}{\sum_{p \in {\Omega_{in}(B_{i})}}{\sum_{j \neq i}{d_{in}( {p,\overset{\hat{}}{B}} )}^{2}}}}{\sum_{i}{\sum_{p \in {\Omega_{in}(B_{i})}}{\sum_{j \neq i}1}}}$

For a match loss, each room bounding box B_(i) should cover the sameregion as the corresponding actual bounding box B_(i)* does, that is,the room bounding box B_(i) should be displayed inside the actualbounding box B_(i)*. Therefore, the match loss may be defined as:

${L_{match}( \{ B_{i} \} )} = {\frac{\sum_{i}{\sum_{p \in {\Omega_{in}(B_{i})}}{d_{in}( {p,B_{i}^{*}} )}^{2}}}{\sum_{i}{❘{\Omega_{in}( B_{i} )}❘}} + \frac{\sum_{i}{\sum_{p \in {\Omega_{in}(B_{i}^{*})}}{d_{in}( {p,B_{i}} )}^{2}}}{\sum_{i}{❘{\Omega_{in}( B_{i}^{*} )}❘}}}$

In the embodiment of the present application, by setting various lossfunctions, various losses are considered, which makes the trained floorplan image generating network finally obtained have better predictionperformance.

In the embodiment of the present application, after obtaining thepredicted floor plan image of the target building outputted by the floorplan image generating network, the method further includes the followingsteps.

Each room bounding box of the target building is aligned with theboundary of the target building.

Adjacent room bounding boxes of the target building are aligned witheach other.

The floor plan image generating network finally outputs the predictedfloor plan image and all room bounding boxes. One problem that may occurto the room bounding boxes is that they may not be well aligned, andsome room bounding boxes may overlap in some regions. Therefore, in thefinal alignment and adjustment step, the grid floor plan image is usedto determine a room type assignment for an overlap region.

Firstly, the room bounding boxes are aligned with the boundary of thetarget building, then the adjacent room bounding boxes are aligned witheach other. More specifically, for each edge of one room bounding box, anearest boundary edge thereof with the same direction, i.e., horizontalor vertical edge, is found. If a distance between the edge of the roombounding box and the nearest boundary edge is less than a given distancethreshold, the edge of the room bounding box is aligned with the nearestboundary edge. Furthermore, adjacent rooms of a room pair are alignedaccording to the spatial relationship in the encoded layout. Forexample, if the room bounding box A is located to the left of the roombounding box B, the right edge of the room bounding box A is alignedwith the left edge of the room bounding box B. In addition, if a lengthdifference of the adjacent edges of two room bounding boxes is less thana given distance threshold, then the adjacent edges of the two roombounding boxes will be aligned, because room bounding boxes arrangedside by side in the floor plan image preferably have aligned walls tominimize the number of corners. A room bounding box may be adjacent todifferent room bounding boxes, therefore, the edges may be updated formany times. To avoid breaking previous optimized alignments, a flag maybe set to indicate whether an edge of the room bounding box has beenupdated. If any edge has been updated, it is fixed, another edge isaligned with the fixed edge. If neither edge is fixed, the two edges areupdated to their average position.

In addition, it is needed to determine room types for the overlapregions between room bounding boxes. To achieve this purpose, each roompair is examined to determine whether two rooms overlap, and a relativeorder of the two rooms is determined by using the generated grid floorplan image. More specifically, for each room pair, the number of pixelsof each room type in the overlap region is calculated, and a room withfewer pixels in the overlap region is drawn first, so that, when thereis an overlap region between two room bounding boxes, the room boundingbox drawn later will overlay the room bounding box drawn earlier, sothat the overlap region is assigned to the room bounding box drawnlater. According to this process, if a first room and a second roomoverlap, and the first room should be drawn before drawing the secondroom, then an image is established by adding a node representing eachroom and a directed edge from the first room to the second room. Then,the goal is to find an order of all nodes in the image, which satisfy anordering constraint imposed by the directed edges. To find such anorder, any node with a zero in-degree is randomly selected firstly, thenthis node and all edges in the rest of the image to which the nodepoints are deleted. Herein, the in-degree of the node is defined as thenumber of directed edges pointing to the node. The nodes having zeroin-degree are removed continuously till the image becomes blank. Itshould be noted that, if there is a cycle in the image, it is impossibleto find a linear order of nodes in the cycle. Therefore, the node withthe smallest in-degree is randomly selected and removed, so as to breakthe cycle.

In the embodiment of the present application, by optimizing thealignment and determining the labels of the overlap regions, the floorplan image finally obtained may display the specific layout of thetarget building more accurately.

Based on the embodiments above, a specific embodiment is provided todescribe an overall process of the floor plan image generating method ofthe present application.

First, the boundary of the target building and the layout constraint ofthe target building inputted by the user (referring to a in FIG. 8 ) areobtained, and the first floor plan images (referring to b in FIG. 8 ),which satisfy the layout constraint inputted by the user, are retrievedfrom the pre-stored floor plan image data set and displayed. Then, aselection is made among the first floor plan images to obtain the secondfloor plan images. The selection includes a selection of the matchingdegree of the boundaries and an adjustment through rotation, which isintended to make the front door direction consistent with the front doordirection of the target building, etc. Then, the layout constraint ofthe second floor plan image is directly applied to the boundary of thetarget building, and the nodes are adjusted to be within the boundary toobtain the multiple layouts of the target building (referring to c inFIG. 8 ). The multiple layouts of the target building are inputted intothe floor plan image generating network, and the predicted floor planimages and the room bounding boxes (referring to d in FIG. 8 ) outputtedby the floor plan image generating network are obtained. Finally, theadjustment of alignment is performed on the predicted floor plan imagesto obtain the adjusted floor plan images (referring to e in FIG. 8 ).

The turning function is used in the selection of the matching degree ofthe boundaries above. For the specific form of the turning function,please refer to FIG. 9 . A left portion of FIG. 9 shows a boundary shapeof the target building, and a right portion of FIG. 9 shows the turningfunction of the boundary of the target building.

In addition, for the process of adjusting and aligning the predictedfloor plan images to obtain the adjusted floor plan images, please referto FIG. 10 . First, the predicted floor plan image (referring to a inFIG. 10 ) is obtained, then the room bounding boxes are aligned with theboundary of the target building (referring to b in FIG. 10 ), finallythe adjacent room bounding boxes are aligned (referring to c in FIG. 10).

In the embodiments of this application, the floor plan image generatingmethod provided in this application may receive user constraints ofdifferent types and different numbers to generate correspondingpredicted floor plan images. FIG. 11 shows a group of views illustratingexemplary results generated by different boundary inputs and differentuser constraint conditions. Views in each row show the result ofdifferent layout constraints applied to the same boundary, while viewsin each column show the result of the same layout constraint applieddifferent boundaries. The selected constraint includes the desirednumbers of three room types including bedroom, bathroom, and balcony.The corresponding constraints for the room quantities are shown at thebottom of each column.

After examining each row in FIG. 11 , it may be known how the generatedpredicted floor plan image satisfies the constraint of the given roomquantities and fits the boundary. The bedrooms, bathrooms, and balconiesof different numbers are generated according to the layout constraints,and the positions of these rooms change so that the floor plan imagebest fits the inputted boundary. The balcony usually has two or threesides and is located at the target building boundary, and reflects atypical design of balcony in an apartment. Therefore, the positions ofthe balconies depend on the inputted boundary. There is a living room inall floor plan images, and the living room is located in a building withsuch a complex boundary, and there may be an extra room.

It may be seen from the results in each column in FIG. 11 , how therooms of the same numbers and same types are distributed differentlywithin buildings with different boundaries. For example, in the thirdcolumn, two bathrooms are sometimes adjacent to each other and sometimesnot adjacent to each other, but are always adjacent to the bedrooms. Inthe fifth column, the balconies are never adjacent to each other andusually located in different positions of the building, which showsdifferent floor plan images.

As may be seen from the above description, the floor plan imagegenerating method provided in this application may generate variousfloor plan images according to different boundaries and different layoutconstraints inputted by the user. Compared with the floor plan imagesgenerated through a simple search by a traditional method, the floorplan images generated by the method of the present application havehigher quality.

It should be understood that, although the steps in the flowcharts ofFIG. 1 to FIG. 8 are shown in sequence indicated by arrows, but thesesteps are not necessarily executed in the order indicated by the arrows.Unless explicitly stated herein, these steps are not necessarilyperformed strictly according to an order, and these steps may beperformed in any other order. Moreover, at least part of the steps inFIG. 1 to FIG. 8 may include multiple steps or multiple stages, andthese steps or stages are not necessarily executed and completed at thesame time, but may be executed at different times. These steps or stagesare not necessarily executed in sequence, but may be performed in turnor alternately with other steps or with at least a portion of steps orstages in the other steps.

In an embodiment of the present application, as shown in FIG. 12 , afloor plan image generating device 1200 is provided, and includes: anacquiring module 1201, an output module 1202, a selecting module 1203, afirst obtaining module 1204, and a second obtaining module 1205.

The acquiring module 1201 is configured to acquire a boundary of atarget building and a layout constraint of the target building, and thelayout constraint includes room types, room quantities, room locations,and adjacency relations between rooms.

The output module 1202 is configured to output multiple first floor planimages according to the layout constraint of the target building.

The selecting module 1203 is configured to select multiple second floorplan images from the multiple first floor plan images, and a matchingdegree between a boundary of each of the multiple second floor planimages and the boundary of the target building satisfies a presetcondition.

The first obtaining module 1204 is configured to, for each of the secondfloor plan images, apply the layout constraint of each of the secondfloor plan images to the boundary of the target building, and obtain alayout of the target building corresponding o each second floor planimage.

The second obtaining module 1205 is configured to, for each layout ofthe target building, input the layout of the target building and theboundary of the target building into a floor plan image generatingnetwork, and obtain a predicted floor plan image of the target buildingoutputted by the floor plan image generating network. The predictedfloor plan image includes a room bounding box of each room in the targetbuilding and a positional relation of each room bounding box in theboundary of the target building.

In an embodiment of the present application, the output module 1202 isspecifically configured to retrieve the multiple first floor plan imagesthat satisfy the layout constraint of the target building from apre-stored floor plan image data set.

In an embodiment of the present application, the selecting module 1203is specifically configured to: obtain turning functions of boundaries ofthe multiple first floor plan images and a turning function of theboundary of the target building; calculate an accumulated differencebetween each of the turning functions of the boundaries of the multiplefirst floor plan images and the turning function of the boundary of thetarget building; and use the multiple first floor plan images eachcorresponding to the accumulated difference smaller than a presetdifference threshold as the multiple second floor plan images.

In an embodiment of the present application, the first obtaining module1204 is specifically configured to: adjust each of the second floor planimages until an angle between a front door direction of each of thesecond floor plan images and a front door direction of the targetbuilding is smaller than a preset angle threshold, and obtain adjustedsecond floor plan images; and for each of the adjusted second floor planimages, apply room types, room quantities, room locations, and adjacencyrelations between rooms of each of the adjusted second floor plan imagesto the boundary of the target building, and obtain the layout of thetarget building corresponding to each of the second floor plan images.

In the embodiment of the present application, the second obtainingmodule 1205 is specifically used to: for each layout of the targetbuilding, input the layout of the target building into an image NN, andobtain a room feature vector corresponding to each room in the layout ofthe target building; input the boundary of the target building into afirst convolutional neural network (CNN) and obtain a boundary featurevector of the target building; correlate the room feature vectorcorresponding to each room with the boundary feature vector to obtain acorrelated feature vector corresponding to each room; input thecorrelated feature vector corresponding to each room into a firstmultilayer perceptron (MLP), and obtains an initial bounding boxcorresponding to each room in the layout of the target building; map thecorrelated feature vector corresponding to each room by using theinitial bounding box corresponding to each room, to obtain a firstfeature image corresponding to each room; combine multiple first featureimages of multiple rooms to obtain a second feature image, then inputsthe second feature image into a second CNN to obtain a grid floor planimage of the target building; and input the multiple correlated featurevectors of the multiple rooms, the multiple initial bounding boxes ofthe multiple rooms and the grid floor plan image into the bounding boxoptimizing network to obtain the predicted floor plan image of thetarget building, and the bounding box optimizing network includes athird CNN, a pooled layer of a region of interest (ROI), and a secondMLP.

In an embodiment of the present application, referring to FIG. 13 ,another floor plan image generating device 1300 is provided. In additionto the modules included in the floor plan image generating device 1200,optionally, the floor plan image generating device 1300 may also includea training module 1206 and an aligning module 1207.

The training module 1206 is configured to obtain a training data set,which includes multiple floor plan image data; train the initial floorplan image generating network by using the training data set to obtainthe trained floor plan image generating network; calculate a loss valueof the trained floor plan image generating network by using a crossentropy loss function, a regression loss function, and a geometric lossfunction; and adjust parameters of the trained floor plan imagegenerating network according to the loss value to obtain the floor planimage generating network.

In an embodiment of the present application, the aligning module 1207 isconfigured to align each room bounding box of the target building withthe boundary of the target building, and align adjacent room boundingboxes of the target building.

Specific limitations on the floor plan image generating device may referto the limitations on the floor plan image generating method above, andwill not be described repeatedly hereinafter. Each module in the abovefloor plan image generating device may be implemented wholly or in partby software, hardware, and combinations thereof. The above modules maybe embedded in or independent of the processor in the computer device inthe form of hardware, or be stored in the memory in the computer devicein the form of software, so that the processor may call and executeoperations corresponding to the above modules.

In an embodiment of the present application, a computer device isprovided, the computer device may be a terminal, and an internalstructure view thereof may be shown in FIG. 14 . The computer deviceincludes a processor, a memory, a communication interface, a displayscreen, and an input device that are connected by a system bus. Theprocessor of the computer device is configured to provide computing andcontrol capabilities. The memory of the computer device includes anon-transitory storage medium, a memory. The non-transitory storagemedium stores an operating system and a computer program. The memoryprovides an environment for running the operating system and thecomputer program in the non-transitory storage medium. The communicationinterface of the computer device is configured for wired or wirelesscommunication with an external terminal, and the wireless communicationcan be realized by WIFI, operator network, near field communication(NFC) or other technologies. The computer program, when executed by theprocessor, implements a floor plan image generating method. The displayscreen of the computer device may be a liquid crystal display screen oran electronic ink display screen. The input device of the computerdevice may be a touch layer covering the display screen, or a button, atrackball or a touchpad provided on a housing of the computer device, oran external keyboard, a trackpad, or a mouse.

Those skilled in the art should understand that the structure shown inFIG. 14 is only a block diagram of part of the structure related to thesolutions of the present application, but not constitutes a limitationon the computer device, to which the solutions of the presentapplication is applied. The specific computer device may include more orless parts than those shown in the figure, or combine some parts, orhave a different arrangement of parts.

In an embodiment, a computer device is provided. The computer deviceincludes a memory and a processor. A computer program is stored on thememory. The processor, when executing the computer program, performs themethod of any one of the embodiments above.

In an embodiment, a non-transitory computer-readable storage medium isprovided, and a computer program is stored on the non-transitorycomputer-readable storage medium. The computer program, when beingexecuted by a processor, performs the method of any one of theembodiments above.

A person of ordinary skill in the art should understand that all or partof the processes in the method of the above embodiments may beimplemented by means of a computer program instructing relevanthardware. The computer program may be stored in a non-volatile computerreadable storage medium. When the computer program is executed, it mayinclude the procedures of the embodiments of the above method. Where,any reference to the memory, the storage, the database or other mediumused in the embodiments provided by the present application may includeat least one of non-transitory memory and transitory memory. Thenon-transitory memory may include read-only memory (ROM), magnetic tape,floppy disk, flash memory, or optical memory. The transitory memory mayinclude random access memory (RAM) or external cache memory. As anillustration but not a limitation, RAM can be in various forms, such asstatic random access memory (SRAM) or dynamic random access memory(DRAM), etc.

The technical features of the above embodiments may be combinedarbitrarily. In order to make the description concise, not all possiblecombinations of the technical features of the above embodiments aredescribed. However, as long as there is no contradiction in thecombination of these technical features, any combination should bewithin the range described in this description.

The above examples are only several embodiments of the presentapplication, and the descriptions thereof are more specific anddetailed, but they should not be understood to be a limitation on thescope of the present invention. It should be noted that, for those ofordinary skill in the art, several modifications and improvements may bemade without departing from the concept of the present application, andall these modifications and improvements fall within the protectionscope of the present application. Therefore, the protection scope of thepresent application shall be subject to the appended claims.

What is claimed is:
 1. A floor plan image generating method, comprising:acquiring a boundary of a target building and a layout constraint of thetarget building, the layout constraint comprising room types, roomquantities, room locations, and adjacency relations between rooms;outputting multiple first floor plan images according to the layoutconstraint of the target building; selecting multiple second floor planimages from the multiple first floor plan images, and a matching degreebetween a boundary of each of the multiple second floor plan images andthe boundary of the target building satisfying a preset condition; foreach of the second floor plan images, applying a layout constraint ofeach of the second floor plan images to the boundary of the targetbuilding, and obtaining a layout of the target building corresponding toeach of the multiple second floor plan images; and for each layout ofthe target building, inputting the layout of the target building and theboundary of the target building into a floor plan image generatingnetwork, and obtaining a predicted floor plan image of the targetbuilding outputted by the floor plan image generating network, whereinthe predicted floor plan image comprises a room bounding box of eachroom in the target building and a positional relation of each roombounding box in the boundary of the target building.
 2. The methodaccording to claim 1, wherein the outputting the multiple first floorplan images according to the layout constraint of the target buildingcomprises: retrieving the multiple first floor plan images that satisfythe layout constraint of the target building from a pre-stored floorplan image data set.
 3. The method according to claim 1, wherein theselecting the multiple second floor plan images from the multiple firstfloor plan images, comprises: obtaining turning functions of boundariesof the multiple first floor plan images and a turning function of theboundary of the target building; calculating an accumulated differencebetween each of the turning functions of the boundaries of the multiplefirst floor plan images and the turning function of the boundary of thetarget building; and using the multiple first floor plan images eachcorresponding to the accumulated difference smaller than a presetdifference threshold as the multiple second floor plan images.
 4. Themethod according to claim 1, wherein, for each of the second floor planimages, applying the layout constraint of each of the second floor planimages to the boundary of the target building, and obtaining the layoutof the target building corresponding to each of the second multiplefloor plan images comprises: adjusting each of the multiple second floorplan images until an angle between a front door direction of each of themultiple second floor plan images and a front door direction of thetarget building is smaller than a preset angle threshold, and obtainingadjusted second floor plan images; and for each of the adjusted secondfloor plan images, applying room types, room quantities, room locations,and adjacency relations between rooms of each of the adjusted secondfloor plan images to the boundary of the target building, and obtainingthe layout of the target building corresponding to each of the multiplesecond floor plan images.
 5. The method according to claim 1, wherein,the for each layout of the target building, inputting the layout of thetarget building and the boundary of the target building into the floorplan image generating network, and obtaining the predicted floor planimage of the target building outputted by the floor plan imagegenerating network, comprise: for each layout of the target building,inputting the layout of the target building into an image neuralnetwork, and obtaining a room feature vector corresponding to each roomin the layout of the target building; inputting the boundary of thetarget building into a first convolutional neural network (CNN), andobtaining a boundary feature vector of the target building; correlatingthe room feature vector corresponding to each room with the boundaryfeature vector to obtain a correlated feature vector corresponding toeach room; inputting the correlated feature vector corresponding to eachroom into a first multilayer perceptron (MLP), and obtaining an initialbounding box corresponding to each room in the layout of the targetbuilding; mapping the correlated feature vector corresponding to eachroom by using the initial bounding box corresponding to each room toobtain a first feature image corresponding to each room; combiningmultiple first feature images of multiple rooms to obtain a secondfeature image, and inputting the second feature image into a second CNNto obtain a grid floor plan image of the target building; and inputtingmultiple correlated feature vectors of the multiple rooms, multipleinitial bounding boxes of the multiple rooms and the grid floor planimage into a bounding box optimizing network to obtain the predictedfloor plan image of the target building, and the bounding box optimizingnetwork comprising a third CNN, a pooled layer of a region of interest,and a second MLP.
 6. The method of claim 1, further comprising:obtaining a training data set, and the training data set comprisingmultiple floor plan image data; training an initial floor plan imagegenerating network by using the training data set to obtain trainedfloor plan image generating network; calculating a loss value of thetrained floor plan image generating network by using a cross entropyloss function, a regression loss function, and a geometric lossfunction; and adjusting parameters of the trained floor plan imagegenerating network according to the loss value to obtain the floor planimage generating network.
 7. The method of claim 1, further comprising:aligning each room bounding box of the target building with the boundaryof the target building; and aligning adjacent room bounding boxes of thetarget building.
 8. The method of claim 1, wherein the acquiring theboundary of the target building and the layout constraint of the targetbuilding comprises: measuring the target building to obtain the boundaryof the target building; and obtaining inputted layout constraint of thetarget building.
 9. The method according to claim 1, wherein theselecting the multiple second floor plan images from the multiple firstfloor plan images, comprises: overlapping and comparing an area enclosedby a boundary of the first floor plan image and an area enclosed by theboundary of the target building; calculating a ratio of an area of anoverlap region to the area enclosed by the boundary of the targetbuilding; and selecting the first floor plan image to be the secondfloor plan image if the ratio exceeds a preset area threshold.
 10. Themethod according to claim 1, wherein the selecting the multiple secondfloor plan images from the multiple first floor plan images, comprises:calculating a perimeter of a boundary and the number of corners of eachof the multiple first floor plan images and a perimeter of the boundaryand the number of corners of the target building; selecting a firstfloor plan image to be the second floor plan image, if a differencebetween the perimeter of the boundary of each of the multiple firstfloor plan images and the perimeter of the boundary of the targetbuilding is less than a preset perimeter difference threshold, and if adifference between the number of corners of each of the multiple firstfloor plan images and the number of corners of the target building isless than a preset number difference threshold.
 11. The method accordingto claim 2, wherein, before the retrieving the multiple first floor planimages that satisfy the layout constraint of the target building fromthe pre-stored floor plan image data set, the method further comprises:acquiring floor plan image data; and encoding the floor plan image data,and storing encoded floor plan image data in the floor plan image dataset.
 12. The method according to claim 3, wherein the accumulateddifference is an area enclosed by a difference between each of theturning functions of the boundaries of the multiple first floor planimages and the turning function of the boundary of the target building,and a normalized accumulated boundary length axis of a coordinatesystem, and the coordinate system is formed by the normalizedaccumulated boundary length and an accumulated angle of each turningpoint in a two-dimensional polygon of the boundary.
 13. The methodaccording to claim 3, wherein the accumulated difference is zero. 14.The method according to claim 11, wherein, the encoding the floor planimage data comprises encoding the room types, the room quantities, theroom locations, and adjacency relations between rooms in the floor planimage.
 15. The method according to claim 4, wherein, after the for eachof the second floor plan images, applying the layout constraint of eachof the second floor plan images to the boundary of the target building,and obtaining the layout of the target building corresponding to each ofthe second multiple floor plan images, the method further comprisesadjusting a node outside the boundary, and the adjusting the nodeoutside the boundary comprises: establishing a bounding box of thetarget building and grids in the bounding box; searching a grid wherethe node outside the boundary is located; and moving the node outsidethe boundary into a nearest grid within the boundary.
 16. The methodaccording to claim 15, wherein, the moving the node outside the boundaryinto the nearest grid within the boundary comprises: if there has been anode already in the nearest grid within the boundary, moving the node inthe grid to a new grid in the same direction; and if a node is moved inthe same direction to the last grid within the boundary, and there stillhas been another node in the last grid, keeping two nodes in the lastgrid, wherein the same direction refers to the same direction alongwhich the node outside the boundary is moved into the nearest gridwithin the boundary.
 17. The method according to claim 5, wherein, theinputting the multiple correlated feature vectors of the multiple rooms,the multiple initial bounding boxes of the multiple rooms and the gridfloor plan image into the bounding box optimizing network to obtain thepredicted floor plan image of the target building comprises: processing,by the third CNN, the grid floor plan image to obtain a third featureimage; inputting each of the multiple initial bounding boxes of themultiple rooms and the third feature image into a pooled layer of aregion of interest to obtain a feature vector with a specific lengthcorresponding to each room; merging and inputting the feature vectorwith the specific length corresponding to each room and each of themultiple correlated feature vectors of the multiple rooms into thesecond MLP to obtain an optimized bounding box corresponding to eachroom; and predicting a visual position relationship of the room boundingbox of each room within the boundary of the target building according tothe optimized bounding box corresponding to each room.
 18. A floor planimage generating device, comprising a processor and a memory, the memoryhaving computer programs stored therein, and the computer programs, whenbeing executed by the processor, causes the processor to perform:acquiring a boundary of a target building and a layout constraint of thetarget building, the layout constraint comprising room types, roomquantities, room locations, and adjacency relations between rooms;outputting multiple first floor plan images according to the layoutconstraint of the target building; selecting multiple second floor planimages from the multiple first floor plan images, and a matching degreebetween a boundary of each of the multiple second floor plan images andthe boundary of the target building satisfying a preset condition; foreach of the second floor plan images, applying a layout constraint ofeach of the second floor plan images to the boundary of the targetbuilding, and obtaining a layout of the target building corresponding toeach of the multiple second floor plan images; and for each layout ofthe target building, inputting the layout of the target building and theboundary of the target building into a floor plan image generatingnetwork, and obtaining a predicted floor plan image of the targetbuilding outputted by the floor plan image generating network, whereinthe predicted floor plan image comprises a room bounding box of eachroom in the target building and a positional relation of each roombounding box in the boundary of the target building.
 19. A computerdevice, comprising a memory and a processor, wherein the memory storescomputer programs, and the processor, when executing the computerprograms, performs steps of the method of claim
 1. 20. A non-transitorycomputer-readable storage medium, on which computer programs are stored,wherein, the computer programs, when being executed by a processor,perform steps of the method of claim 1.